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REAL PARTY IN INTEREST 

The real party in interest is Koninklijke Philips Electronics N.V., a corporation 
of The Netherlands having an office and a place of business at Groenewoudseweg 1, 
Eindhoven, Netherlands 5621 BA. Koninklijke Philips Electronics N.V. is the parent 
company of the assignee of record U.S. Philips Corporation, a Delaware corporation 
having an office and a place of business at 345 Scarborough Road, Briarcliff Manor, 
New York, 10510-8001. 
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RELATED APPEALS AND INTERFERENCES 

To the best of Appellants' knowledge and belief, there are no related appeals or 
interferences. 
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STATUS OF CLAIMS 

Claims 1-6 and 8-21 are pending in this application. Claims 1-6 and 8-21 are 
rejected in the Final Office Action that mailed December 29, 2004. This rejection was 
upheld in an Advisory Action that mailed March 23, 2005. Claims 1-6 and 8-21 are 
the subject of this appeal. 
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STATUS OF AMENDMENTS 

An Amendment After Final Action was filed Febmary 9, 2005 in response to 
the Final Office Action. The Advisory Action upheld the rejection in response to that 
amendment. This Appeal Brief is in response to the Final Office Action that rejected 
Claims 1-6 and 8-21 and the Advisory Action that upheld that rejection. 



5 



Patent 

Application No. 09/819,971 
Appeal Brief 

SUMMARY OF CLAIMED SUBJECT MATTER 

The present invention, for example as claimed in independent Claims 1, 8 and 
15, relates to a method, apparatus and system, respectively, for reacquiring a target in 
a video tracking system. 

The video tracking system shown in Fig. 1 of U.S. Patent Application Serial 
No. 09/819,971 includes a camera 102 for providing video image data of a scene 
having a desired target 106 to be tracked. The system further includes a display 1 10a 
for displaying the video image data of the scene 104 from the camera 102. An input 
device (e.g., joystick) is used to select a desired target image 106. After selecting a 
desired target 106a, the automated video tracking system is switched to an automatic 
mode to initiate a tracking sequence to automatically track the selected desired target 
106a. The tracking system 116 generates a model of a target 106a to be tracked that 
can be used to locate the target 106a in successive frames of the video image data. 
Once a model of the target 106a is generated, a controller 1 1 8 is then instructed by the 
automated tracking system 1 16 to move the camera 102 (via PZT motors 108) to keep 
the selected target 106 centered in the field of view of the camera 102. For example, 
see par. 27 of U.S. Patent Application Serial No. 09/819,971. The controller 118 
determines the amount of camera movement necessary to keep the target 106 centered 
in the field of view of the camera 102. If the video tracking system has encountered a 
period of difficulty and has lost the selected target 106a or may be in jeopardy of 
losing the selected target 106a. For example, see par. 33 of U.S. Patent Application 
Serial No. 09/819,971. A period of difficulty may be characterized as a particular 
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situation which is difficult for the automatic video tracking system to resolve. For 
example, multiple people passing by the same location at the same time as the selected 
target, the operator may take over for the system for the time it takes for the target to 
pass the source of difficulty. It is noted that if no period of difficulty is encountered, 
the automated video tracking system continues to automatically track the selected 
target until the method is terminated, the selected target 106a leaves the scene 104, or 
it is no longer desired to track the selected target 106a. If the operator perceives a 
difficulty has or is about to be encountered, the method proceeds by switching fi'om 
automatic mode to manual mode. For example, see par. 33 of U.S. Patent Application 
Serial No. 09/819,971. Once in manual mode, the operator reacquires the desired 
target. It is preferred that the target is reacquired in a simpler manner as compared to 
the way it is initially selected, namely, by centering the desired target 106a in the 
monitor's display 1 10a of the scene 104 by manipulating the joystick 1 12 or other 
input device. For example, see par. 34 of U.S. Patent AppUcation Serial No. 
09/819,971. Once the target is reacquired, the automated video tracking system is 
switched back to the automatic mode at step 212 where the desired target is 
automatically tracked without initiating a new tracking sequence. It is important for 
the operator to quickly reacquire the desired target 106a so that a new tracking 
sequence is not initiated after the target 106a is reacquired and automatic tracking 
restarted. Preferably, automatic tracking restarts by either a manual instruction fi'om 
the user, as discussed above, or preferably automatically by releasing control of the 
joystick 1 12 or other input device used to reacquire the desired target 106a. After 
switching back to automatic mode, the automatic video tracking system continues to 
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track the reacquired target 106a until the target leaves the scene, tracking of the 
reacquired target is no longer desired, or another area of difficulty is encountered by 
the automatic tracking system,. 

The method includes the step of selecting a desired target 106a to be tracked. 
For example, see element 202 of FIG. 2 of U.S. Patent AppUcation Serial No. 
09/819,971 . When the target 106a is selected, a computer model is built to represent 
the appearance of that target 106a. During tracking of the target 106a, whenever the 
tracker finds a part of the image that matches to the target model, it preferably 
computes a number which represents how well the target 106a matches the model. 
This number can vary for example from 0% match to 100% match. Where the 100% 
indicates that the target matches the model completely. This value is called the 
confidence value. A control can also be provided to indicate a threshold value for the 
confidence. See for example, the text at par. 36 of U.S. Patent AppUcation Serial No. 
09/819,971. The method fiirther includes the step of switching the automated video 
tracking system to an automatic track mode to initiate a tracking sequence to 
automatically track the selected desired target. For example, see element 204 of FIG. 2 
of U.S. Patent Application Serial No. 09/819,971. The method fiirther includes the 
step of, during the automatic track mode, the automated video tracking system 
providing a warning to a user indicating that said automatic track mode is about to 
fail whenever the calculated confidence value falls below a pre-determined threshold 
confidence value. For example, see claim 7 of U.S. Patent AppUcation Serial No. 
09/819,971 . The method fiirther includes the step of switching the automated video 
tracking system from an automatic mode to a manual mode if the automated video 
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tracking system encounters a period of difficulty in tracking the tracked selected 
desired target. For example, see element 208 of FIG. 2 of U.S. Patent Application 
Serial No. 09/819,971. The method further includes the step of reacquiring the 
selected desired target in manual mode in response to and during said period of 
difficulty. For example, see element 210 of FIG. 2 of U.S. Patent Application Serial 
No. 09/819,971. The method fiirther includes the step of switching back fi-om the 
manual mode to the automatic mode without initiating a new tracking sequence. For 
example, see element 212 of FIG. 2 of U.S. Patent AppUcation Serial No. 09/819,971. 
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GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

Whether Claims 1-6, 8-9, 11, 13, 16 and 18 and 20 of U.S. Patent Application 
Serial No. 09/819,971 are obvious under 35 U.S.C. §103(a) over U.S. Patent No. 
5,982,420 to Ratz ("Ratz") in view of U.S. Patent No. 5,867,584 to Hu ("Hu"). The 
Appellant respectfully wishes the Board to address the patentability of Claims 1, 8 and 
15 and further Claims 2-6, 9-14 and 16-21, as depending thereon, based on the 
requirements of Claims 1, 8 and 15, respectively. This position is provided for the 
specific and stated purpose of simplifying the current issue on appeal. However, the 
Appellant herein specifically wishes to reserve the right to argue and address the 
patentability of each of the further claims at a later date should the separately 
patentable subject matter of those claims later become an issue. Accordingly, this 
limitation of the subject matter presented for appeal herein, specifically limited to 
discussions of the patentability of Claims 1, 8 and 15 is not intended as a waiver of 
Appellant's right to argue the patentability of the further claims and claim elements at 
that later time. 
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ARGUMENT 

Claims 1-6, 8-9, 1 1, 13, 16 and 18 and 20 are said to be unpatentable over Ratz 
in view ofHu. 

The Ratz Patent 

Ratz shows a stand-alone video tracking device that is easily added to existing 
remotely controlled camera systems without extensive modification. See Ratz, Col. 1, 
lines 59-65. The automatic tracking device comprises means for generating steering 
signals that are applied to detecting circuits to cause the camera to track an object. See 
Ratz, Col. 2, lines 16-18. 

An operator is shown a white rectangle box 28 as an operator prompt to inform 
the operator that the object 14 he/she is attempting to track has enough contrast for 
him/her to lock onto (see Ratz, Col. 12, lines 15-18). If any video image or image 
information related to object 14 is detected the counter is reset (see Ratz, Col. 12, lines 
18-20). If two consecutive fields occur with no detectable object in the window, the 
white rectangle 28 is extinguished. As soon as video or image information related to 
the object 14 is detected again, the rectangular box 28 is illuminated (see Ratz, Col 
12, lines 24-25). 

An operator initiates tracking with a manual override switch located on an 
operator console 20 (see Ratz, Col. 12, lines 33-34). A trigger switch on the joystick 
can be the source of a manual override signal 44. Steering signals are not accepted by 
the camera directing circuits unless the trigger is squeezed (i.e., manual mode). When 
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the trigger switch is released the camera is locked in that position (i.e., automatic 
mode) (see Ratz, Col. 12, line 40). 

More particularly, the autotracker 18 of Ratz couples to an existing tracking 
device by intercepting the trigger switch conmiand signal and connecting it to signal 
path 44, as shown in FIG. 7. If the autotracker 18 is not locked on a target the window 
will be frozen in the center of the display screen. When the trigger is released the 
autotracker 18 window 28 will begin following any object that was in the window 28 
at the time the trigger is released (see Ratz, Col. 12, lines 54-55). When the trigger on 
a joystick 22 is squeezed, the manual override signal of the operator's console 20 
allows an operator to slew the camera as normal (i.e., manual mode). This squeezing 
action will allow him/her to move an object 14 he/she wishes to track into the window. 
Releasing the trigger switch locks the autotracker 18 on the object 14. 

If the autotracker 18 has successfully locked on a target the steering voltages 
will now come from the autotracker 18 instead of the operator, (see Ratz, CoL 12, 
lines 62-63). The autotracker 18 will disable itself if the object disappears, (see Ratz, 
Col. 13, line 1). 

The Examiner's First Position 

The Examiner maintains in the Final Office Action at page 4, that Ratz does 
show disengaging the autotracker if the object encounters a period of difficulty. See 
Ratz, Col. 13, lines 1-5 wherein it is stated: 
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'' The autotracker 18 will disable itself if the object disappears. I f 64 consecutive fields of data 
related to the object 14 occur with no video or image information (about one second) latch U12B is 
clocked low by counter UIO of FIG. 7, allowing the manual override 46 to return to logic zero and 
forcing the digital-to-analog converter outputs of FIG, 8 and FIG. 9 to zero volts with the presence of 
the HOME command. " [Emphasis Added] 

The Appellant's First Position 

With regard to the Examiner's assertion that . .Ratz does show disengaging 
the autotracker if the object encounters a period of difficulty. . .". While Ratz may in 
fact show disengagement, the Examiner fails to appreciate the difference in the terms 
"disengagement" and "period of difficulty". As argued in the appellant's response 
filed on February 9, 2005, it was asserted that the specification teaches that a "period 
of difficulty", can be, for example, having the confidence value fall temporarily below 
a pre-defined threshold, as taught in step (d) of Claim 1 . This is clearly distinguishable 
firom the more permanent condition of "disengagement" or "disablement", as taught in 
Ratz at Col. 13, lines 1-5, wherein it is stated: The autotracker 18 will disable itself if 
the object disappears, " 

By contrast, as argued in the final Office Action, the invention is directed to a 
capability for re-acquiring a target in an automated video tracking system. Ratz only 
addresses the most extreme situation of an object disappearing. The invention is 
directed to the more common situation of a tracking system experiencing a temporary 
track loss or failure and providing a solution by temporarily switching fi*om automatic 
to manual track mode, having an operator intervene in the manual mode to correct the 
temporary track loss and switching back firom the manual to automatic track mode. 
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The method steps involve the constmction of a computer model of a target 106 
to be tracked to represent the appearance of a target 106a. During the tracking of the 
target 106a, whenever the tracker finds a part of the image that matches to the target 
model, it computes a number which represents how well the target 106a matches the 
model, i.e., a confidence value (see Claim 1, step (c)). A control can also be provided 
to indicate a threshold value for the confidence value. Thus, should the model match 
the target with less than this threshold value, the operator is preferably wamed by a 
signal of some form that the confidence is lower than the threshold and the tracking is 
about to fail (see Claim 1, step (d)). In such a situation, the operator may take over for 
the system for the time it takes for the target to pass the source of difficulty by 
reacquiring the target, after which control is given back to the automatic tracking 
system (see Claim 1, step (e)). 

As argued in the final office action, the period of difficulty in tracking the 
target automatically is a temporary state that is overcome by operator intervention, i.e., 
switching from an automatic to a manual mode. This is clearly distinguishable from 
the autotracker disabling itself if the object disappears, as disclosed in Ratz at Col. 13, 
lines 1-5. 

In a phone conversation with the Examiner on April 26, 2005, the appellant 
proposed to amend step (e) of Claim 1 to more clearly recite the fact that the period of 
difficulty is a "temporary" state. In particular, the appellant proposed the following 
amendment to step (e) of claim 1 : 
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. . .(e) switching the automated video tracking system from an automatic mode 
to a manual mode if the automated video tracking system encounters a temporary 
period of difficulty in tracking the tracked selected desired target,. . . 

The Examiner discussed this proposal with his supervisor and concluded that 
Ratz allegedly teaches step (e) at Col. 12, lines 15-27. The Appellant did not have an 
opportunity to discuss the Examiner's conclusion. The Appelant respectfully points 
out, , however, that upon a close reading of Col. 12, lines 15-27 that Ratz does not in 
fact teach a so-called "temporary disablement". Instead, Col. 12, lines 15-27 teaches 
the case where an operator has yet to lock onto a target. This is clearly distinguishable 
from the method taught in independent Claims 1, 8 and 15, where an object is already 
being tracked in automatic mode and must be temporarily switched to manual mode to 
overcome a temporary period of difficulty. 

The Examiner's Second Position 

The Examiner maintains in the Final Office Action at page 4, that Hu discloses 
"during the automatic track mode, the automated video tracking system calculates a 
confidence value indicating a degree of correlation between the tracked target and a 
previously constructed computer model. The Examiner cites Hu at Col. 3, lines 35-40, 
wherein the confidence value is the quality and at Col. 4, lines 1-4, wherein the 
previously constructed computer model is the file from which the user has to open to 
begin the tracking procedure. 
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The Appellant^s Second Position 

With regard to the Examiner's assertion that Hu discloses "during the 
automatic track mode, the automated video tracking system calculates a confidence 
value indicating a degree of correlation between the tracked target and a previously 
constructed computer model 

In accordance with the principles of the invention, as disclosed at par. 36 of 
U.S. Patent Application Serial No. 09/819,971, when the target 106a is initially 
selected, a computer model is built to represent the appearance of that target 106a. 
During tracking of the target 106a, whenever the tracker finds a part of the tracked 
video image that matches to the target model, it preferably computes a number which 
represents how well the target 106a matches the model. 

In the Final Office Action, the Examiner maintains that the previously 
constructed computer model is the file fi-om which the user has to open to begin the 
tracking procedure. Hu discloses at Col. 4, lines 1-10, that the user opens the video file 
to select an object the user wishes to track and defines an object window around the 
object and then selects the track button which instructs the system to automatically 
track the object in subsequent fi-ames. 

The system of Hu is directed to automatically tracking objects through a video 
sequence. For example, see Hu in the Abstract. The video sequence relates to 
interactive multimedia applications, interactive television and games. For example, 
see Hu at Col. 1, lines 6-10. Assuming that the previously constructed computer 
model is the file fi-om which the user has to open to begin the tracking procedure, Hu 
cannot teach or disclose the comparison of a computer model with a tracked video 
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image of the target, as recited in the independent claims as amended because each 
frame of the opened file displays the computer model for purposes of tracking. That 
is, there is no teaching or disclosure in Hu of a comparison being made between the 
computer model displayed in each frame of the file with a tracked video target because 
in Hu, the tracked target is the computer model according to the Examinerr's 
interpretation. Hu only teaches that if the tracking system fails to track the object in a 
particular frame, the system will warn the user and wait for further instructions from 
the user. For example, see Hu at Col. 4, lines 14-17. 



17 



Patent 

Application No. 09/819,971 
Appeal Brief 

CONCLUSION 

Claims 1-6 and 8-21 are patentable over Ratz, alone or in view of Hu. 
Thus the Examiner's rejection of Claims 1-6 and 8-21 should be reversed. 

Respectfully submitted, 

Dicran Halajian, ^ 
Reg. No. 39,703 
Attorney for Applicant 
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APPENDIX A 

CLAIMS ON APPEAL 

1 . (Previously Amended) A method for reacquiring a target in an automated video 
tracking system, the method comprising the steps of: 

(a) selecting a desired target to be tracked; 

(b) switching the automated video tracking system to an automatic 
track mode to initiate a tracking sequence to automatically track the selected desired 
target; 

(c) during said automatic track mode, the automated video tracking 
system calculating a confidence value indicating a degree of correlation between the 
tracked video image of the target and a previously constructed computer model of said 
tracked target; 

(d) during said automatic track mode, the automated video tracking 
system providing a v^aming to a user indicating that said automatic track mode is 
about to fail whenever said calculated confidence value falls below a pre-determined 
threshold confidence value; 

(e) switching the automated video tracking system from an automatic 
mode to a manual mode if the automated video tracking system encounters a period of 
difficulty in tracking the tracked selected desired target; 

(f) reacquiring of the selected desired target in manual mode in 
response to and during said period of difficulty; and 

(g) switching the automated video tracking system to the automatic 
mode for automatic tracking of the reaquired selected desired target without initiating 
a new tracking sequence. 

2. (Original) The method of claim 1, wherein step (a) comprises 
centering the desired target in a display of a scene including the desired target. 
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3. (Original) The method of claim 1, wherein step (b) comprises 

releasing control of an input device used to select the desired target. 

4. (Original) The method of claim 1, wherein step (c) comprises 
controlling an input device used to select the desired target. 

5. (Original) The method of claim 1, wherein step (d) comprises 
centering the desired target in a display of a scene including the desired target. 

6. (Original) The method of claim 1, wherein step (e) comprises 
releasing control of an input device used to reacquire the desired target. 



7. (Cancelled) 

8. (Previously Amended) An apparatus for reacquiring a target in an 
automated video tracking system, the apparatus comprising: 

selecting means for selecting a desired target to be tracked; 

mode switching means for switching the automated video tracking 
system to and from one of an automatic mode to initiate a tracking sequence after 
target selection to automatically track the selected desired target and a manual mode; 

calculation means for calculating a confidence value indicating a 
degree of correlation between the video image of the tracked target and a previously 
constructed computer model of said tracked target; 

waming means for providing a warning to a user indicating that said 
automatic track mode is about to fail whenever said calculated confidence value falls 
below a pre-determined threshold confidence value; 
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reacquiring means for reacquiring the desired target in manual mode in 

response to and during the automated video tracking system encountering a period of 

difficulty in tracking the desired target; 

wherein after reacquiring the desired target the automated video 

tracking system is switched back to automatic mode without initiating a new tracking 

sequence. 

9. (Original) The apparatus of claim 8, wherein the selecting means 
comprises an input device for centering the desired target in a display of a scene 
including the desired target. 

10. (Original) The apparatus of claim 9, further comprising: 

a video camera for capturing video image data of a scene including the 

desired target; 

pan and tilt camera motors for controlling a pan and tilt, respectively of 
the video camera; and 

a video display for displaying the video image data; 

wherein the input device is a joystick operatively connected to the pan 
and tilt motors such that movement of the joystick controls the movement of the 
camera through the pan and tilt motors. 

1 1 . (Original) The apparatus of claim 8, wherein the mode selecting 
means comprises an input device where the automated video tracking system is 
switched to automatic mode by controlling an input device used to select the desired 
target and the automated video tracking system is switched to manual mode by 
releasing control of the input device. 
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12. (Original) The apparatus of claim 1 1, further comprising: 

a video camera for capturing video image data of a scene including the 

desired target; 

pan and tilt camera motors for controlling a pan and tilt, respectively of 
the video camera; and 

a video display for displaying the video image data; 

wherein the input device is a joystick operatively connected to the pan 
and tilt motors such that movement of the joystick controls the movement of the 
camera through the pan and tilt motors. 

13. (Original) The apparatus of claim 8, wherein the reacquiring means 
comprises an input device for centering the desired target in a display of a scene 
including the desired target. 

14. (Original) The apparatus of claim 13, further comprising: 

a video camera for capturing video image data of a scene including the 

desired target; 

pan and tilt camera motors for controlling a pan and tilt, respectively of 
the video camera; and 

a video display for displaying the video image data; 

wherein the input device is a joystick operatively connected to the pan 
and tih motors such that movement of the joystick controls the movement of the 
camera through the pan and tilt motors. 

15. (Previously Amended) An automated video tracking system for 
tracking and reacquiring a target, the automated video tracking system comprising: 

a video camera for capturing video image data of a scene including a 

desired target; 
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pan and tilt camera motors for controlling a pan and tilt, respectively of 
the video camera; 

a video display for displaying the video image data; 

selecting means for selecting the desired target to be tracked; 

mode switching means for switching the automated video tracking 
system to and from one of an automatic mode to initiate a tracking sequence after 
target selection to automatically track the selected desired target and a manual mode; 

calculation means for calculating a confidence value indicating a 
degree of correlation between the video image of the tracked target and a previously 
constructed computer model of said tracked target; 

warning means for providing a warning to a user indicating that said 
automatic track mode is about to fail whenever said calculated confidence value falls 
below a pre-determined threshold confidence value; 

reacquiring means for reacquiring the desired target in manual mode in 
response to and dxiring the automated video tracking system encountering a period of 
difficulty in tracking the desired target; 

wherein after reacquiring the desired target the automated video 
tracking system is switched back to automatic mode without initiating a new tracking 
sequence. 

16. (Original) The automated video tracking system of claim 15, 
wherein the selecting means comprises an input device for centering the desired target 
in the display. 

17. (Original) The automated video tracking system of claim 16, 
wherein the input device is a joystick operatively connected to the pan and tilt motors 
such that movement of the joystick controls the movement of the camera through the 
pan and tilt motors. 
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18. (Original) The automated video tracking system of claim 15, wherein 
the mode selecting means comprises an input device where the automated video tracking 
system is switched to automatic mode by controlling an input device used to select the 
desired target and the automated video tracking system is switched to manual mode by 
releasing control of the input device. 

19. (Original) The automated video tracking system of claim 18, wherein 
the input device is a joystick operatively connected to the pan and tilt motors such that 
movement of the joystick controls the movement of the camera through the pan and tilt 
motors. 

20. (Original) The automated video tracking system of claim 15, wherein 
the reacquiring means comprises an input device for centering the desired target in a 
display of a scene including the desired target. 

21. (Original) The automated video tracking system of claim 20, wherein 
the input device is a joystick operatively connected to the pan and tilt motors such that 
movement of the joystick controls the movement of the camera through the pan and tiU 
motors. 
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APPENDIX B 



Evidence on Appeal 



None 
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APPENDIX C 



Related Proceedings on 



None 



